Selecting Multiple Web Adverts - a Contextual Multi-armed Bandit with State Uncertainty

We present a method to solve the problem of choosing a set of adverts to display to each of a sequence of web users. The objective is to maximise user clicks over time and to do so we must learn about the quality of each advert in an online manner by observing user clicks. We formulate the problem as a novel variant of a contextual combinatorial multi-armed bandit problem. The context takes the form of a probability distribution over the user's latent topic preference, and rewards are a particular nonlinear function of the selected set and the context. These features ensure that optimal sets of adverts are appropriately diverse. We give a flexible solution method which combines submodular optimisation with existing bandit index policies. User state uncertainty creates ambiguity in interpreting user feedback which prohibits exact Bayesian updating, but we give an approximate method that is shown to work well

Mixed-strategy learning with continuous action sets

Motivated by the recent applications of game-theoretical learning to the design of distributed control systems, we study a class of control problems that can be formulated as potential games with continuous action sets. We propose an actor-critic reinforcement learning algorithm that adapts mixed strategies over continuous action spaces. To analyse the algorithm we extend the theory of finite-dimensional two-timescale stochastic approximation to a Banach space setting, and prove that the continuous dynamics of the process converge to equilibrium in the case of potential games. These results combine to give a provablyconvergent learning algorithm in which players do not need to keep track of the controls selected by other agents

REX:a development platform and online learning approach for Runtime emergent software systems

Conventional approaches to self-adaptive software architectures require human experts to specify models, policies and processes by which software can adapt to its environment. We present REX, a complete platform and online learning approach for runtime emergent software systems, in which all decisions about the assembly and adaptation of software are machine-derived. REX is built with three major, integrated layers: (i) a novel component-based programming language called Dana, enabling discovered assembly of systems and very low cost adaptation of those systems for dynamic re-assembly; (ii) a perception, assembly and learning framework (PAL) built on Dana, which abstracts emergent software into configurations and perception streams; and (iii) an online learning implementation based on a linear bandit model, which helps solve the search space explosion problem inherent in runtime emergent software. Using an emergent web server as a case study, we show how software can be autonomously self-assembled from discovered parts, and continually optimized over time (by using alternative parts) as it is subjected to different deployment conditions. Our system begins with no knowledge that it is specifically assembling a web server, nor with knowledge of the deployment conditions that may occur at runtime

Literature Themes from Five Decades of Agricultural Communications Publications

The discipline of agricultural communications has been developing for nearly two centuries. As the discipline has adapted, professional organizations such as the American Association of `Agricultural College Editors (AAACE) and the Association for Communication Excellence in Agriculture, Natural Resources, and Life and Human Sciences (ACE) have published literature representative of the topics and issues that have impacted the discipline through magazines and journals such as the AAACE, ACE Quarterly, and the Journal of Applied Communications (JAC). The purpose of this study was to review the literature published in AAACE, ACE Quarterly, and JAC from 1968-2015 to identify primary and secondary literature themes. There were 13 emergent themes identified. The most prolific primary theme identified was Channel Development, Use or Research while the most prolific secondary theme identified was Educating Professionals. A count of the number of articles classified as “professional development” and “research” revealed a shift in the focus in the journal outlets. In earlier years, the discipline focused mainly on professional development articles (AAACE and ACE Quarterly), but transitioned almost completely to research (JAC). This research acknowledges that the discipline has experienced significant literary shifts and provides a recommendation for further research in audience analysis of the literature coming from the journals of the discipline

Bandit learning in concave N-player games

This paper examines the long-run behavior of learning with bandit feedback in non-cooperative concave games. The bandit framework accounts for extremely low-information environments where the agents may not even know they are playing a game; as such, the agents' most sensible choice in this setting would be to employ a no-regret learning algorithm. In general, this does not mean that the players' behavior stabilizes in the long run: no-regret learning may lead to cycles, even with perfect gradient information. However, if a standard monotonicity condition is satisfied, our analysis shows that no-regret learning based on mirror descent with bandit feedback converges to Nash equilibrium with probability 1. We also derive an upper bound for the convergence rate of the process that nearly matches the best attainable rate for single-agent bandit stochastic optimization

Using J-K-fold Cross Validation to Reduce Variance When Tuning NLP Models

K-fold cross validation (CV) is a popular method for estimating the true performance of machine learning models, allowing model selection and parameter tuning. However, the very process of CV requires random partitioning of the data and so our performance estimates are in fact stochastic, with variability that can be substantial for natural language processing tasks. We demonstrate that these unstable estimates cannot be relied upon for effective parameter tuning. The resulting tuned parameters are highly sensitive to how our data is partitioned, meaning that we often select sub-optimal parameter choices and have serious reproducibility issues. Instead, we propose to use the less variable J-K-fold CV, in which J independent K-fold cross validations are used to assess performance. Our main contributions are extending J-K-fold CV from performance estimation to parameter tuning and investigating how to choose J and K. We argue that variability is more important than bias for effective tuning and so advocate lower choices of K than are typically seen in the NLP literature, instead use the saved computation to increase J. To demonstrate the generality of our recommendations we investigate a wide range of case-studies: sentiment classification (both general and target-specific), part-of-speech tagging and document classification

The Lantern Vol. 44, No. 1, Fall 1977

• Onto My Love • Saturday Midnight • Michelle • Today • Firefly • Black Midnight • Bamboo Arms • Caesaropapism • A Day In My Life • I Only • For Stephen • April 18, 1958 to July 15, 1977 with Emphasis on July 15 • Ode to Little Sisters • Privacy Warning • For Susan, Someone I Used to Know • A Parting on the Night of June 26th • Infant\u27s Universehttps://digitalcommons.ursinus.edu/lantern/1111/thumbnail.jp

Effects of flood hazard visualization format on house purchasing decisions

We investigated how decision-making is affected by the visual presentation of flood hazard information. We exposed participants to different formats of flood hazard information while they simulated selecting a property to purchase. We compared three flood hazard formats: (i) maps currently used by the UK Environment Agency, (ii) tables that present flood level and frequency information and (iii) graphical representations depicting the level-frequency combination using a cartoon house image as a physical referent. In the experiment participants were presented, via computer screen, side-by-side information about two houses in a series of trials. Participants made a forced choice preference judgement between 108 different pairs of houses to indicate which they would purchase. Our findings indicate that when hazard information is presented in map format, individuals are less accurate in selecting lower-hazard houses, compared to when the same information is presented as a graphic representation of a house or as a table. © 2018, © 2018 Informa UK Limited, trading as Taylor & Francis Group

Population and seascape genomics of a critically endangered benthic elasmobranch, the blue skate Dipturus batis

The blue skate (Dipturus batis) has a patchy distribution across the North-East Atlantic Ocean, largely restricted to occidental seas around the British Isles following fisheries-induced population declines and extirpations. The viability of remnant populations remains uncertain, and could be impacted by continued fishing and bycatch pressure and the projected impacts of climate change. We genotyped 503 samples of D. batis, obtained opportunistically from the widest available geographic range, across 6,350 single nucleotide polymorphisms (SNPs) using a reduced-representation sequencing approach. Genotypes were used to assess the species’ contemporary population structure, estimate effective population sizes, and identify putative signals of selection in relation to environmental variables using a seascape genomics approach. We identified genetic discontinuities between inshore (British Isles) and offshore (Rockall and Faroe Island) populations, with differentiation most pronounced across the deep waters of the Rockall Trough. Effective population sizes were largest in the Celtic Sea and Rockall, but low enough to be of potential conservation concern among Scottish and Faroese sites. Among the 21 candidate SNPs under positive selection was one significantly correlated with environmental variables predicted to be affected by climate change, including bottom temperature, salinity, and pH. The paucity of well annotated elasmobranch genomes precluded us from identifying a putative function for this SNP. Nevertheless, our findings suggest that climate change could inflict a strong selective force upon remnant populations of D. batis, further constraining its already restricted habitat. Furthermore, the results provide fundamental insights on the distribution, behaviour, and evolutionary biology of D. batis in the North-East Atlantic that will be useful for the establishment of conservation actions for this and other critically endangered elasmobranchs

On-Orbit Results From the NASA Time-Resolved Observations of Precipitation Structure and Storm Intensity With a Constellation of Smallsats (TROPICS) Mission

The NASA TROPICS Earth Venture (EVI-3) CubeSat constellation mission will provide nearly all-weather observations of 3-D temperature and humidity, as well as cloud ice and precipitation horizontal structure, at high temporal resolution to conduct high-value science investigations of tropical cyclones. TROPICS will provide rapid-refresh microwave measurements (median refresh rate better than 60 minutes for the baseline mission) over the tropics that can be used to observe the thermodynamics of the troposphere and precipitation structure for storm systems at the mesoscale and synoptic scale over the entire storm lifecycle. The TROPICS constellation mission comprises four 3UCubeSats (5.4 kg each) in two low-Earth orbital planes. Each CubeSat contains a Blue Canyon Technologies bus and a high-performance radiometer payload to provide temperature profiles using seven channels near the 118.75 GHz oxygen absorption line, water vapor profiles using three channels near the 183 GHz water vapor absorption line, imagery in a single channel near 90 GHz for precipitation measurements (when combined with higher resolution water vapor channels), and a single channel at 205 GHz that is more sensitive to precipitation-sized ice particles. TROPICS spatial resolution and measurement sensitivity is comparable with current state-of-the-art observing platforms. Two dedicated launches (two spacecraft per launch) for the TROPICS constellation mission on Rocket Lab Electron vehicles occurred in 2023 (May 8 and May 26) to place the spacecraft in 32.75-degree inclined orbits at 550 km altitude. Data will be downlinked to the ground via the KSAT-Lite ground network. NASA\u27s Earth System Science Pathfinder (ESSP) Program Office approved the separate TROPICS Pathfinder mission, which launched on June 30, 2021, in advance of the TROPICS constellation mission as a technology demonstration and risk reduction effort. The TROPICS Pathfinder mission has provided an opportunity to checkout and optimize all mission elements prior to the primary constellation mission and is still operating nominally